The present invention relates to an information processing apparatus, such as a lockstep fault tolerant computer, that simultaneously processes the same instructions in a plurality of clock-synchronized computer modules therein, and more particularly, to an information processing apparatus that speedily synchronizes a computer module, which has been out of synchronism with the other computer modules and isolated from the operation, with other computer modules.
A conventional lockstep fault tolerant computer has a plurality of computer modules which simultaneously execute the same instructions. In the fault tolerant computer, one of the computer modules may operate differently from the other computer modules because of a failure or some other causes. Upon detecting a computer module that operates differently from the other computer modules, in other words, on finding a computer module which is out of lockstep synchronism, the lockstep fault tolerant computer once puts the detected computer module out of the operation.
Causes which make the computer module be out of the lockstep synchronism vary. A course of reaction to be taken for the computer module, which is out of the lockstep synchronism, depends on the cause. One of the causes, which makes the computer module be out of the lockstep synchronism, may be a permanent failure that occurs within the computer module. The permanent failure is not a temporary disturbance or a failure that recovers by the computer module itself, but a failure requiring repairs. A computer module, in which a permanent failure occurs, is usually taken out of the lockstep fault tolerant computer and, instead of that module, another healthy computer module is installed.
Another potential cause, which makes the computer module be out of the lockstep synchronism, may be a lack of synchronism that the operation timing does not synchronize temporarily with the other computer modules because of manufacturing variations of the computer modules. Yet another potential cause may be temporary malfunction of a memory in the computer module affected by an influence such as an a ray. In those causes like a lack of synchronism or temporary malfunction, which does not cause a permanent failure, the computer module need not be replaced.
If the permanent failure occurs, the faulty computer module is replaced and the replaced computer module is joined to and synchronized with the other computer modules. If there is no permanent failure, the computer module is rejoined to and resynchronized with the other computer modules. The operation to make a disconnected computer module rejoin the other computer modules is a resynchronization. When the conventional lockstep fault tolerant computer resynchronizes with the computer module which was out of the lock step synchronism, the conventional lockstep fault tolerant computer copies a memory of the computer module, which is to be rejoined, from a memory of another computer module which is in the lockstep synchronism. There joined computer module there after executes the same operations with the other computer modules.
A conventional lockstep fault tolerant computer forces all computing modules stop and copies the whole contents of memory of the joined or rejoined computer module from another computer module being in the lockstep synchronism when joining or rejoining the computing module. This allows all the computing modules to have completely the same internal state. A conventional lockstep fault tolerant computer is forced to stop long time to join or rejoin the computer module. This is because it takes a long time to copy the whole contents of the memory in the computer module. Especially, as memory size in the computer module increases, time to copy the whole content of the memory in the computer module increases.